Neuroecology and psychological modularity.

the activity difference δ′ t and the regular temporal difference δ t never substantially affects the plasticity of a synapse. This happens, for instance, if there is only in fact one single reward right at the end of the trial [e.g. r t =0, t < T; r(T)=1]. In a simplification of the version of this that Rao and Sejnowski consider, r(T)=1 comes from an action potential, caused by a privileged input to the postsynaptic cell, backpropagating up its dendritic tree. This makes positive the activity difference associated with pre-synaptic events initiated at time t=T−1 in a trial, thus engendering increases in synaptic efficacy. These, in turn, reduce the activity difference, by increasing P T−1 , until the difference reaches 0. This process can lead to the prior pre-synaptic events causing the post-synaptic cell to spike in a preditive manner. Rao and Sejnowski further suggest a specialized inhibitory connection architecture [18], which allows the predictive spike to cancel out the predicted spike (thus eliminating the effect of the difference between δ t and δ′ t). In the converse case, what happens if indeed δ′ t is used in the learning rule rather than δ t ? I don't know of compelling computational analyses of this case, other than the obvious point that the resulting learning rule looks like a correlational learning rule between the stimuli and the differences in successive outputs. Rao and Sejnowski face the even trickier problem of making the learning rules work in the face of biophysically realistic timescales for synaptic currents and membrane potentials and the like. The most dangerous problem that arises is instability, that the learning rule can make the synaptic efficacies rise without bound. This happens when the biophysical mechanism for propagating information around the post-synaptic cell (backpropagating action potentials) lasts over a longer time scale than that involved in the derivative P t+1 −P t. That can make the learning rule operate more like a regular correlational learning rule, and these are notoriously unstable. Synaptic saturation is suggested as a possible fix, although one might worry about a consequent loss of synaptic selectivity. Altogether, the notion that temporally asymmetric Hebbian learning rules are best seen in predictive rather than correlational terms has been taken in various interesting directions. Rao and Sejnowski usefully add to our armoury of ways of approaching such rules, and remind us of an essential Yogic truth.